miri: prune some atomic operation and raw pointer details from stacktrace #98674

RalfJung · 2022-06-29T14:22:37Z

Since Miri removes track_caller frames from the stacktrace, adding that attribute can help make backtraces more readable (similar to how it makes panic locations better). I made them only show up with cfg(miri) to make sure the extra arguments induced by track_caller do not cause any runtime performance trouble.

This is also testing the waters for whether the libs team is okay with having these attributes in their code, or whether you'd prefer if we find some other way to do this. If you are fine with this, we will probably want to add it to a lot more functions (all the other atomic operations, to start).

Before:

error: Undefined Behavior: Data race detected between Atomic Load on Thread(id = 2) and Write on Thread(id = 1) at alloc1727 (current vector clock = VClock([9, 0, 6]), conflicting timestamp = VClock([0, 6]))
    --> /home/r/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:2594:23
     |
2594 |             SeqCst => intrinsics::atomic_load_seqcst(dst),
     |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Data race detected between Atomic Load on Thread(id = 2) and Write on Thread(id = 1) at alloc1727 (current vector clock = VClock([9, 0, 6]), conflicting timestamp = VClock([0, 6]))
     |
     = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
     = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
             
     = note: inside `std::sync::atomic::atomic_load::<usize>` at /home/r/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:2594:23
     = note: inside `std::sync::atomic::AtomicUsize::load` at /home/r/.rustup/toolchains/miri/lib/rustlib/src/rust/library/core/src/sync/atomic.rs:1719:26
note: inside closure at ../miri/tests/fail/data_race/atomic_read_na_write_race1.rs:22:13
    --> ../miri/tests/fail/data_race/atomic_read_na_write_race1.rs:22:13
     |
22   |             (&*c.0).load(Ordering::SeqCst)
     |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

After:

error: Undefined Behavior: Data race detected between Atomic Load on Thread(id = 2) and Write on Thread(id = 1) at alloc1727 (current vector clock = VClock([9, 0, 6]), conflicting timestamp = VClock([0, 6]))
  --> tests/fail/data_race/atomic_read_na_write_race1.rs:22:13
   |
22 |             (&*c.0).load(Ordering::SeqCst)
   |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Data race detected between Atomic Load on Thread(id = 2) and Write on Thread(id = 1) at alloc1727 (current vector clock = VClock([9, 0, 6]), conflicting timestamp = VClock([0, 6]))
   |
   = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
   = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
           
   = note: inside closure at tests/fail/data_race/atomic_read_na_write_race1.rs:22:13

rustbot · 2022-06-29T14:22:43Z

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then
please comment with @rustbot label +T-libs-api -T-libs to tag it
appropriately. If this PR contains changes to any unstable APIs please edit
the PR description to add a link to the relevant API Change
Proposal
or create one
if you haven't already. If you're unsure where your change falls no worries,
just leave it as is and the reviewer will take a look and make a decision to
forward on if necessary.

Examples of T-libs-api changes:

Stabilizing library features
Introducing insta-stable changes such as new implementations of existing
stable traits on existing stable types
Introducing new or changing existing unstable library APIs (excluding
permanently unstable features / features without a tracking issue)
Changing public documentation in ways that create new stability guarantees
Changing observable runtime behavior of library APIs

rust-highfive · 2022-06-29T14:22:44Z

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

Mark-Simulacrum · 2022-06-30T01:04:48Z

The new trace is definitely nicer. I'm not sure how I feel about cfg'ing these purely for miri - I think we can panic in some of these in practice, so it might be useful regardless? On the other hand, for low-level primitives like this the extra register usage might be a real pain.

I'm going to nominate for the libs team to answer the broader question ("This is also testing the waters for whether the libs team is okay with having these attributes in their code, or whether you'd prefer if we find some other way to do this. If you are fine with this, we will probably want to add it to a lot more functions (all the other atomic operations, to start).").

I personally feel like these attributes are OK -- where possible, we should avoid divergence between miri and non-miri, but for performance reasons I'm OK with miri being special cased. It may make sense to make that cfg something other than miri in the future (e.g., slower_but_nicer_std) or something -- we have similar things around for example RefCell reporting the location of the 'other' borrow when panicking -- but that's a broader question and can easily wait until there's a good story around users specifying that kind of option.

saethlin · 2022-07-02T23:56:24Z

I don't want to sidetrack this issue, but your response touches on something broader and I'm not entirely sure where to discuss.

slower_but_nicer_std... ...but that's a broader question and can easily wait until there's a good story around users specifying that kind of option.

There is a similar question around assert_unsafe_precondition, we already have users of the standard library debug assertions, and there is some interest in an option that makes them emit something like a panic message. So I agree this is not urgent, but I think these things will require -Zbuild-std and waiting until that's stable seems like too long to me.

Mark-Simulacrum · 2022-07-03T00:05:03Z

I don't think we necessarily need build-std; we could ship precompiled std/core for a small set of configurations if we wanted to. It's just something someone would need to propose a way to access (through Cargo and/or rustup).

But yes, I agree that there's a number of similar assertions that might be nice to give users access to.

m-ou-se · 2022-07-20T16:46:28Z

This was discussed in a recent libs team meeting, and we're fine with these #[cfg_attr(miri, track_caller)] attributes. If a more general mechism ever arrives or if we ever want to get rid of them again, they're very easy to grep for and replace/remove.

RalfJung · 2022-07-20T20:43:25Z

Awesome! I added all the track_caller that I can think of for the first round.

@Mark-Simulacrum this is ready for review.
@rustbot ready

Mark-Simulacrum · 2022-07-23T16:59:35Z

@bors r+ rollup=iffy

bors · 2022-07-23T16:59:36Z

📌 Commit 13877a9 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

RalfJung · 2022-07-23T17:21:59Z

@bors r-
delaying until after the next nightly, to make sure we get a Miri published with that nightly.

RalfJung · 2022-07-23T21:06:58Z

@bors r=Mark-Simulacrum

bors · 2022-07-23T21:06:59Z

📌 Commit 13877a9 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

bors · 2022-07-24T06:46:49Z

⌛ Testing commit 13877a9 with merge 35a0617...

bors · 2022-07-24T09:15:56Z

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 35a0617 to master...

rust-highfive · 2022-07-24T09:16:27Z

📣 Toolstate changed by #98674!

Tested on commit 35a0617.
Direct link to PR: #98674

💔 miri on windows: test-pass → test-fail (cc @RalfJung @oli-obk).
💔 miri on linux: test-pass → test-fail (cc @RalfJung @oli-obk).

@RalfJung

Tested on commit rust-lang/rust@35a0617. Direct link to PR: <rust-lang/rust#98674> 💔 miri on windows: test-pass → test-fail (cc @RalfJung @oli-obk). 💔 miri on linux: test-pass → test-fail (cc @RalfJung @oli-obk).

rust-timer · 2022-07-24T10:32:29Z

Finished benchmarking commit (35a0617): comparison url.

Instruction count

Primary benchmarks: no relevant changes found
Secondary benchmarks: 😿 relevant regression found

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	1.2%	1.2%	1
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	N/A	N/A	0
All 😿🎉 (primary)	N/A	N/A	0

Max RSS (memory usage)

Results

Primary benchmarks: 😿 relevant regressions found
Secondary benchmarks: 😿 relevant regressions found

	mean¹	max	count²
Regressions 😿 (primary)	3.2%	3.4%	2
Regressions 😿 (secondary)	2.9%	3.1%	3
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	N/A	N/A	0
All 😿🎉 (primary)	3.2%	3.4%	2

Cycles

Results

Primary benchmarks: 😿 relevant regression found
Secondary benchmarks: 🎉 relevant improvements found

	mean¹	max	count²
Regressions 😿 (primary)	3.1%	3.1%	1
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	N/A	N/A	0
Improvements 🎉 (secondary)	-3.2%	-5.4%	6
All 😿🎉 (primary)	3.1%	3.1%	1

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

…Simulacrum add miri-track-caller to more intrinsic-exposing methods Follow-up to rust-lang#98674: I went through the Miri test suite to find more functions that would benefit from Miri backtrace pruning, and this is what I found. Basically anything that just exposes a potentially-UB intrinsic to the user should get this treatment.

add miri-track-caller to more intrinsic-exposing methods Follow-up to rust-lang/rust#98674: I went through the Miri test suite to find more functions that would benefit from Miri backtrace pruning, and this is what I found. Basically anything that just exposes a potentially-UB intrinsic to the user should get this treatment.

rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Jun 29, 2022

rust-highfive assigned Mark-Simulacrum Jun 29, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 29, 2022

m-ou-se removed the I-libs-nominated Nominated for discussion during a libs team meeting. label Jul 20, 2022

RalfJung added 2 commits July 20, 2022 16:34

miri: prune some atomic operation details from stacktrace

2b269ca

prune raw pointer read and write methods from Miri backtraces

13877a9

RalfJung force-pushed the miri-stacktrace-pruning branch from 2f636ef to 13877a9 Compare July 20, 2022 20:42

rustbot added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 20, 2022

RalfJung removed the S-waiting-on-team Status: Awaiting decision from the relevant subteam (see the T-<team> label). label Jul 20, 2022

RalfJung mentioned this pull request Jul 22, 2022

make some Miri backtraces more pretty rust-lang/portable-simd#295

Merged

RalfJung changed the title ~~miri: prune some atomic operation details from stacktrace~~ miri: prune some atomic operation and raw pointer details from stacktrace Jul 22, 2022

RalfJung mentioned this pull request Jul 23, 2022

add a flag to print a diagnostic when an outdated value is returned from an atomic load rust-lang/miri#2424

Merged

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 23, 2022

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jul 23, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jul 23, 2022

bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 24, 2022

bors merged commit 35a0617 into rust-lang:master Jul 24, 2022

rustbot added this to the 1.64.0 milestone Jul 24, 2022

rust-highfive mentioned this pull request Jul 24, 2022

miri no longer builds after rust-lang/rust#98674 #99664

Closed

bors mentioned this pull request Jul 24, 2022

Stabilize const_ptr_read #97320

Merged

RalfJung deleted the miri-stacktrace-pruning branch July 24, 2022 13:14

RalfJung mentioned this pull request Jul 24, 2022

add miri-track-caller to more intrinsic-exposing methods #99690

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

miri: prune some atomic operation and raw pointer details from stacktrace #98674

miri: prune some atomic operation and raw pointer details from stacktrace #98674

RalfJung commented Jun 29, 2022

rustbot commented Jun 29, 2022

rust-highfive commented Jun 29, 2022

Mark-Simulacrum commented Jun 30, 2022

saethlin commented Jul 2, 2022

Mark-Simulacrum commented Jul 3, 2022

m-ou-se commented Jul 20, 2022

RalfJung commented Jul 20, 2022

Mark-Simulacrum commented Jul 23, 2022

bors commented Jul 23, 2022

RalfJung commented Jul 23, 2022 •

edited

Loading

RalfJung commented Jul 23, 2022

bors commented Jul 23, 2022

bors commented Jul 24, 2022

bors commented Jul 24, 2022

rust-highfive commented Jul 24, 2022

rust-timer commented Jul 24, 2022

miri: prune some atomic operation and raw pointer details from stacktrace #98674

miri: prune some atomic operation and raw pointer details from stacktrace #98674

Conversation

RalfJung commented Jun 29, 2022

rustbot commented Jun 29, 2022

rust-highfive commented Jun 29, 2022

Mark-Simulacrum commented Jun 30, 2022

saethlin commented Jul 2, 2022

Mark-Simulacrum commented Jul 3, 2022

m-ou-se commented Jul 20, 2022

RalfJung commented Jul 20, 2022

Mark-Simulacrum commented Jul 23, 2022

bors commented Jul 23, 2022

RalfJung commented Jul 23, 2022 • edited Loading

RalfJung commented Jul 23, 2022

bors commented Jul 23, 2022

bors commented Jul 24, 2022

bors commented Jul 24, 2022

rust-highfive commented Jul 24, 2022

rust-timer commented Jul 24, 2022

Instruction count

Max RSS (memory usage)

Cycles

Footnotes

RalfJung commented Jul 23, 2022 •

edited

Loading